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VTbe  study  tested  the  feasibility  of  computer-based  training  (CBT)  for 
students  with  below  average  academic  skills  and  evaluated  a job  perfor- 
mance aid  used  in  recipe  conversion  for  the  Mess  Management  Specialist 
School.  There  were  20  students  in  each  of  3 groups:  two  CBT  experimental 

groups  and  a control  group.  One  CBT  group  received  the  job  aid  and  the 
other,  the  traditional  math  as  taught  in  the  school  but  on-line.  The ^ 
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control  group  received  classroom  training  from  an  instructor  using  tradi- 
tional math.  The  CBT  groups  required  significantly  less  training  time  than 
the  control  group  (4.4  hours  vs.  26.5  hours)  while  maintaining  high  scores 
on  the  final  test  which  was  criterion-referenced.  However,  the  CBT  job 
aid  group  performed  significantly  poorer  than  the  no  job  aid  and  classroom 
groups.  The  student  questionnaire  indicated  that  the  typical  student  gave 
a high  rating  to  the  CBT  instruction.  It  is  suggested  that  individualized 
CBT  i6  efficient  in  terms  of  reduced  training  time. 
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SUMMARY 


Problem 

Although  computer-based  training  (CBT)  techniques  have  been  shown  to 
be  effective,  they  have  been  applied  mostly  to  trainees  average  and  above 
average  in  academic  aptitude.  Thus,  there  was  a need  to  develop  effective 
teaching  techniques  for  below  average  trainees  and  for  special  curriculum 
areas  in  which  there  are  training  problems. 

Obi ective 

The  goals  of  the  study  were  (1)  to  determine  the  feasibility  of  computer- 
based  training  (CBT)  for  students  with  below  average  academic  skills,  (2) 
to  evaluate  the  benefit  of  a job  performance  aid  designed  to  facilitate  effi- 
cient task  performance  and  reduce  training  requirements,  and  (3)  to  assess 
the  attitude  of  students  toward  CBT. 

Approach 

A total  of  60  students  from  the  Mess  Management  Specialist  Class  "A" 
School  participated  in  the  study.  This  school  was  selected  because  students 
do  not  have  to  meet  stringent  entrance  requirements.  The  students  utilized 
were  in  their  second  week  of  a heavily  math-oriented  course  on  recipe  con- 
version, in  which  50  percent  of  the  students  scored  below  75  percent  on 
the  area  final  examination  and  performed  poorly  in  the  lab  situation. 

The  students  were  divided  into  three  groups:  two  CBT  experimental  groups 

and  a control  group.  The  control  group  received  traditional  training, 
which  consisted  of  1 week  of  classroom  lectures.  The  experimental  groups 
received  self-paced  individualized  CBT  materials  presented  in  a learner 
control  mode.  One  CBT  group  was  taught  recipe  conversion  using  mathematical 
solutions,  and  the  second,  using  a job  aid  designed  to  decrease  errors 
and  to  reduce  time  required  to  adjust  a recipe.  On  completion  of  training, 
CBT  groups  completed  an  attitude  questionnaire.  The  final  test  for  all 
groups  consisted  of  actual  recipe  conversion  problems. 

Findings  ' 

1.  CBT  resulted  in  significantly  reducing  training  time  (about  4. A 
hours  versus  26.5  for  classroom  instruction)^  All  groups  scored  high  on 
the  final  test,  although  the  Job  Aid  group  (X  **  88)  scored  significantly 
lower  than  the  No  Job  Aid  group  (X  - 94)  and  the  classroom  group  (X  » 95). 

2.  The  job  aid  used  in  the  experimental  treatment  did  not  produce 
the  expected  savings  in  training  time  or  reduction  of  recipe  conversion 
errors. 

3.  The  student  questionnaire  indicated  that  students  adapted  well 

to  learner  control  and  rated  CBT  as  an  "outstanding"  model  of  instruction. 
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Conclusions 


The  data  strongly  suggest  that  self-paced  individualized  CBT  can  be  a 
useful  instructional  methodology  for  low  aptitude  trainees.  Training  time 
is  shortened  without  degrading  performance  compared  to  traditional  class- 
room instruction.  However,  if  CBT  is  to  be  applied  in  the  Mess  Manage- 
ment Specialist  curriculum,  a method  is  needed  to  coordinate  this  flexi- 
bility in  training  time  with  the  realities  of  the  lab  (galley)  situation. 

Recommendations 


1.  Use  of  the  job  aid  should  be  taught  separately  from  theory  on 
recipe  conversion. 

2.  The  CBT  lesson  developed  using  mathematical  solutions  should  be 
used  as  a math  remedial  tool  by  the  MS  school. 
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INTRODUCTION 


Prob  1 em 


A goal  of  Navy  training  Is  to  minimize  training  time  while  maintaining 
or  enhancing  performance.  This  goal  lias  typically  been  accomplished  by 
providing  individualized  or  self-paced  instruction  through  application  of 
computer-based  training  (C3T) . However,  in  the  past,  CBT  techniques  have 
been  applied  primarily  to  highly  technical  Navy  "A"  schools  (i.e.,  elec- 
tronics), which  use  stringent  procedures  in  selecting  trainees  (Ford, 
Slough,  6,  Hurlock,  1972).  Thus,  there  was  a need  to  develop  effective 
teaching  techniques  for  below  average  trainees  and  for  special  curriculum 
areas  in  which  there  are  training  problems. 

Purpose 


The  primary  objective  of  this  study  was  to  determine  the  feasibility 
of  CBT  for  students  with  below  average  academic  skills,  as  evidenced  by 
scores  obtained  on  the  Navy  Basic  Test  Battery.  A secondary  objective 
was  to  evaluate  the  use  of  job  aids,  because  tiiey  had  been  considered 
quite  important  in  the  instruction  of  lower  aptitude  personnel  (Main, 
Harrigan,  & Hooprieh,  1971).  These  job  aids  were  originally  developed 
for  the  Commissaryman/Steward  ratings,  which  have  since  been  combined  to 
form  the  Mess  Management  Specialist  (MS)  rating.  No  experimental  evalua- 
tion was  made  of  these  job  aids  at  the  time  of  their  development. 

Finally,  tills  study  attempted  to  assess  student  attitudes  in  regard 
to  CBT  technology,  sometimes  referred  to  as  computer-aided  instruction 
(CA1) . There  has  been  some  research  (Cole,  1971)  indicating  that  "under- 
educated"  adults  react  negatively  to  such  instruction.  Cole  found  that 
subjects  in  his  study  were  not  able  to  handle  the  complexities  of  the  IBM 
1500  system  student  terminal.  Students  felt  that  the  response  modes  were 
too  complicated  and  were  upset  by  the  random  delays  and  frequent  system 
failures.  In  a follow-up  questionnaire,  they  indicated  that  they  did  not 
like  the  system  and  that  an  hour  of  CAI  a day  was  the  maximum  desired. 

Cole  concluded  that  a smooth  running  system  was  needed  to  eliminate  nega- 
tive student  response.  In  contrast,  NAVPERSRANDCF.N  studies  on  the  IBM  1500 
system  witii  higher  ability  students  found  that  most  students  preferred  CAI 
over  classroom  instruction,  scored  higher  on  tests,  and  trained  45  percent 
faster  than  classroom  controls  (Ford,  Slough,  & Hurlock,  1972). 
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METHOD 


Sul)  jccts 

A total  of  60  Navy  trainees  at  the  Mess  Management  Specialist  (MS) 

Class  "A"  School,  Naval  Training  Center,  San  Diego  were  randomly  selected 
over  a 10-week  testing  period  to  participate  in  the  study.  Although  eli- 
gibility for  training  in  "A"  schools  normally  is  partially  dependent  on 
scores  obtained  on  the  Navy  Basic  Test  Battery  (e.g.,  GCT  and  ARI),  MS 
schools  have  no  formal  cutoff  score  for  admittance.  As  a result,  the 
academic  skills  of  MS  students  may  vary  considerably. 

The  MS  course  runs  for  approximately  8 weeks.  The  students  selected 
were  in  their  second  week,  and  were  being  instructed  in  the  math-oriented 
recipe  conversion  area.  This  task  involves  changing  a standard  recipe 
for  100  servings  to  accommodate  different  numbers  of  servings  while  main- 
taining the  relative  proportions  of  the  ingredients.  Experience  has  shown 
that  recipe  conversion  is  an  area  in  which  trainees  experience  consistent 
learning  difficulties  and  instructors  report  the  most  pedagogical  problems. 
To  illustrate,  at  the  time  this  study  commenced,  records  indicated  that 
at  least  half  of  MS  students  were  scoring  below  75  percent  on  the  area 
final  examination  and  also  were  performing  poorly  in  the  laboratory  situa- 
tion (galley),  which  is  the  real  criterion  measure  for  the  school. 

Experimental  Design 

The  subjects  were  divided  into  three  groups  of  20  students  each — two 
experimental  groups,  to  receive  computer-based  training  (CBT),  and  a con- 
trol group,  to  recieve  traditional  instructor-classroom  training.  One 
CBT  group  was  to  use  the  conventional  long-hand  math  solution  approach; 
and  the  other,  job  performance  aids,  as  represented  with  computer  graphics 
and  a hard  copy  of  the  job  aid.*  It  was  expected  that  job  aids  would  not 
only  decrease  the  time  required  to  solve  recipe  conversion  problems  but 
also  reduce  errors. 

Table  1,  gives  mean  GCT  and  ARI  test  scores  for  each  group.  The  school 
has  no  formal  cutoff  score  for  admittance  and  there  is  a wide  range  of  com- 
bined GCT  and  ARI  scores.  Over  35  percent  of  the  students  scored  below  the 
generally  accepted  minimum  of  100.  The  subjects  therefore  represented  a 
population  of  students  with  average/below  average  academic  skills,  at  least 
as  compared  to  electronics  students,  whose  GCT  + ARI  scores  average  about 
120  (Lahey,  Hurlock,  & McCann,  1973). 


’Original  plans  were  to  present  the  job  aid  cards  using  a microfiche 
slide  selector  rather  than  by  graphics  displays.  Accordingly,  photos  were 
taken  of  artwork  representing  the  job  aid  cards,  which  used  the  largest 
letters  possible  and  various  colored  backgrounds,  while  still  maintaining 
the  required  detail.  However,  when  the  microfiche  was  projected  on  a plasma 
tube  with  black  background,  the  numbers  were  not  legible  because  there  was 
not  enough  contrast. 
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Table  1 

Means  and  Standard  Deviations  for  Background  Scores  by  Group 


CBT— 

CBT— 

_ 

Job  Aid 

No  Job 

Aid 

Control 

Test 

Mean 

SD 

Me  an 

SD 

Mean 

SD 

CCT 

53.9 

7.53 

52.6 

6.31 

52.2 

5.45 

ARI 

50.65 

4.46 

51.5 

6.2 

52.35 

6.58 

Hardware 

The  two  individualized  courses  on  recipe  conversion  were  presented  via 
PLATO  IV,  an  instructional  time-sharing  system.  Student  terminals  in  San 
Diego  were  connected  to  a CDC  Cyber  73  computer  at  the  University  of  Illinois/ 
Urbana  by  voice-grade  telephone  lines.  The  terminals  consist  of  a plasma 
display  panel  with  a keyboard  and  touch  panel  interface.  Stifle  (1970, 

1971)  presents  a more  detailed  description  of  the  PLATO  IV  system  and 
terminal . 

Lesson  Material 


All  the  recipes  used  in  the  services  are  contained  on  Armed  Forces 
Recipe  Service  (AFRS)  cards  and  based  on  a standard  of  100  portions.  These 
recipes  must  be  adjusted  to  accommodate  different  numbers  of  servings;  the 
numbers  of  people  served  per  meal  changes  routinely,  requiring  either  an 
increase  or  decrease  in  quantities  of  food  prepared.  Also,  they  are  some- 
times adjusted  to  use  up  surplus  amounts  of  food.  The  adjustment  of  the 
recipe  to  meet  these  needs  is  called  recipe  conversion. 

The  computer-assisted  instruction  (CAI)  lessons  for  the  two  experimental 
groups  implemented  an  early  version  of  a learner  control  "driver"  lesson, 
which  was  developed  for  the  PLATO  IV  system  (Lahey,  Crawford,  & Hurlock, 
1976)  and  patterned  after  strategies  employed  by  Merrill  (1973).  Although 
the  learner  control  driver  has  the  course  strategy  "designed-in"  as  an 
integral  part  of  the  system,  the  exact  course  content  must  be  added  by 
the  author.  Under  the  learner  control  theory,  the  student  may  choose  his 
own  approach  to  control  of  lesson  strategy. 

Lahey,  Hurlock,  and  McCann  (1973),  in  their  study  on  CBT  using  elec- 
tronics students,  demonstrated  that  these  students  learned  as  well  In  a 
learner  control  situation  as  they  did  in  a program  control  situation  where 
a pretest  determined  the  learning  sequence.  Fredericks  (1976),  in  a related 
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study,  found  that  a student  control  group  performed  significantly  faster 
than  a program  control  group.  Their  test  scores  were  also  higher  than  those 
of  the  program  control  group,  hut  the  difference  was  not  significant. 


It  should  be  noted  that  learner  control  may  involve  different  procedures 
in  different  studies.  For  example,  in  the  Lahey,  et  al.  study,  the  student 
had  only  sequence  control.  That  is,  he  was  free  to  choose  the  sequence 
of  the  objectives  he  wanted  to  study  from  a list  of  course  objectives. 
However,  once  he  had  made  this  decision,  he  was  led  through  a tutorial 
sequence.  In  Fredericks'  (197b)  study,  students  had  some  control  over 
the  amount  of  training  recieved.  In  the  present  effort,  CBT  students 
were  allowed  both  sequence  and  strategy  control;  that  is,  they  could  choose 
the  sequence,  difficulty  level  (easy,  medium,  hard),  the  type  of  presen- 
tation (rule,  example,  practice),  and  the  quantity  desired.  Figure  1 shows 
an  example  of  a practice  problem  on  line. 

As  indicated  previously,  the  CAI  lessons  were  designed  to  teach  two 
different  methods  of  adjusting  recipes.  One  lesson — for  the  No  Job  Aid 
(NJA)  group — was  an  individualized  version  of  the  traditional  long-hand 
math  approach  now  used  in  the  MS  school.  The  other— -for  the  Job  Aid  (JA) 
group — presented  recipe  adjustment  solutions  using  a job  aid  presented  on 
line  and  off  line,  which  had  previously  been  designed  to  reduce  training 
requirements  and  optimize  task  performance.  At  the  top  of  Figure  1 is  a 
graphical  example  of  one  job  aid  card  used  in  conversion. 

Figure  2 shows  the  training  index  used  for  the  no  job-aid  version  of 
the  CBT.  This  index  is  the  initial  contact  the  student  has  with  the  lesson 
material,  following  an  introductory  unit  that  familiarizes  him  with  the 
keyboard  and  special  keys  required  to  enable  him  to  move  between  presenta- 
tion modes. 

Procedure 


During  the  first  day  of  the  recipe  conversion  class,  the  actual  measuring 
equipment  was  introduced  and  demonstrated.  Examples  of  the  AFRS  recipe 
cards  were  distributed,  and  the  card  format  was  explained  in  detail.  At 
the  beginning  of  the  second  day,  the  AO  students  assigned  to  the  experi- 
mental groups  were  told  to  report  to  the  PLATO/CAI  laboratory.  The  20 
students  In  the  control  group  remained  in  the  classroom  and  continued  their 
st  ud 1 es . 

When  the  CAI  students  arrived  in  the  PIATO  lab,  the  proctor  gave  them 
a presentation  describing  the  goals  of  the  project  and  randomly  assigned 
them  identification  numbers  that  determined  which  version  of  the  CAI  lessons 
they  were  to  receive,  introduced  them  to  the  terminal  keyboard,  and  taught 
them  how  to  use  the  special  keys  required  for  operating  within  the  learner 
control  format.  The  appropriate  CAI  lesson  material  was  initiated  when  the 
student  indicated  his  desire  to  begin.  After  he  had  finished  the  lesson, 
he  completed  an  attitude  questionnaire,  followed  by  an  alternate  form  of 
the  school  final  test,  which  completed  the  CRT  lab  portion  of  the  study. 
Students  were  then  al lowed  to  return  to  the  classroom.  The  final  test — 
for  all  groups — consisted  of  actual  recipe  conversion  problems. 
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RESULTS 


Comparison  of  CAT  and  Control  Groups 

Table  2,  which  compares  the  training  time  required  and  final  test  scores 
obtained  by  the  three  groups,  shows  that  the  two  CBT  groups  required  less 
than  one-fifth  the  training  time  required  by  the  control  group.  This  was 
highly  significant  for  both  CBT  groups  (t  = 56.03,  p < .001  and  t = 48.17, 
p <■  .001).  However,  the  Job  Aid  (JA)  group  performed  significantly  poorer 
on  the  final  exam  than  the  No-Job  Aid  (NJA)  or  the  control  groups  (t  = 2.13 
and  t = 2.47,  p < .05).  This  finding  was  exactly  opposite  to  what  had  been 
predicted.  It  may  be  that  the  job  aid  failed  to  enhance  learning  and  per- 
formance because  it  needs  to  be  redesigned  from  a human  factors  standpoint. 
Many  students  had  trouble  learning  to  use  the  job  aid;  rather  than  a help. 

It  became  a task  that  was  required  in  addition  to  solving  recipe  conversion 
problems.  In  other  words,  in  the  case  of  the  job  aid  CRT  lesson,  two  new 
things  were  actually  being  taught:  how  to  understand  the  theory  of  receipe 

conversion  and  how  to  use  the  job  aid  to  actually  convert  a recipe.  As  a 
result,  the  job  aid  lesson  contained  nine  objectives;  and  the  no  job  aid 
lesson,  only  six.  Thus,  it  appears  that  the  job  aid  should  not  be  intro- 
duced when  a student  is  learning  about  the  theory  of  recipe  conversion. 
Ratfier,  it  could  be  introduced  later  as  a tool  for  an  already  knowledge- 
able cook  (just  as  a hand  calculator  would  be  another  alternative  to  man- 
ually doing  division/multiplication). 


Table  2 

Means  and  Standard  Deviations  for  Training  Time 
In  Hours  and  School  Final  Test  Scores  in  Recipe  Conversion  Area 


CRT- 
Job  Aid 
Group 

CBT — 
No  Job 
Group 

Aid 

Control 

Group 

Measures 

Mean 

sn 

Mean 

SD 

Mean  SD 

Time  (Hours) 

4.44 

1.86 

4.34 

2.07 

26.5**  0a 

Final  Score  (./') 

87.7* 

16.28 

94.4 

4.92 

95.5  4.75 

'Fixed  schedule  for  all  Ss. 
*p  « .05 

**p  < .001 


9 


S t ikIimi  t At  t i t ud  t' 


1 


Since  the  lesson  strategy  was  based  on  learner  control,  where  the  initia- 
tive lies  with  the  student,  it  was  important  to  determine  whether  the  CBT 
students  felt  comfortable  making  the  decisions  necessary  to  progress  through 
the  material.  Thus,  one  of  the  questions  in  the  attitude  questionnaire 
asked  the  students  to  rate  their  ability  to  "figure  out  what  to  do  next, 
without  being  told."  They  were  to  respond  by  using  a five-point  scale, 
ranging  from  "very  easy"  to  "very  difficult."  Results  showed  that  both 
CBT  groups  were  comfortable  with  learner  control.  Eight  members  (40%) 
of  the  .JA  group  responded  with  "very  easy":  three  (15%),  "easy";  and  nine 
(45%),  "somewhat  easy."  For  the  NJA  group,  two  (10%)  responded  with  "very 
easy";  ten  (50%),  "easy";  and  six  (30%),  "somewhat  easy."  Only  one  NJA 
group  member  felt  that  learner  control  was  "somewhat  difficult,"  and  one 
failed  to  respond. 

Another  question  asked  the  students  to  rate  computer-based  training  on 
a five-point  scale  ranging  from  "outstanding"  to  "poor."  Again,  results 
were  positive.  Sixteen  members  (80%)  of  the  JA  group  felt  that  CBT  was 
"outstanding,"  and  four  (20%),  "above  average."  For  the  NJA  group,  twelve 
( A 0% ) rated  CBT  as  "outstanding";  and  five  (25%),  "above  average."  Only 
one  NJA  group  member  felt  that  CBT  was  "fair,"  and  two  failed  to  respond. 

PI.ATO  System  Reliability 

Seventy-nine  percent  of  the  CBT  students  experienced  line  errors;  that 
is,  problems  with  the  transmission  of  material  over  telephone  lines.  These 
errors  appeared  to  the  students  either  as  overlays  of  two  lines,  which  were 
impossible  to  read,  or  as  interruptions,  when  the  screen  went  blank  and 
the  student  had  to  sign  back  onto  the  lesson.  Because  the  instructional 
materials  were  presented  under  a learner  control  mode,  the  problems  were 
especially  frustrating  to  the  CBT  students.  They  reported  that  the  line 
errors  resulted  in  (1)  interruptions  in  train  of  thought  and  loss  of  con- 
tinuity, because  of  being  sent  back  to  the  index  by  the  learner  driver, 
and  (2)  frustration  and  anger  with  the  system  at  being  forced  to  "pace 
through"  several  problems  to  get  back  to  the  point  of  interruption. 
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DISCUSSION  AND  CONCLUSIONS 


Computer-based  training  (CBT)  appears  to  be  effective  for  the  average 
and  below  average  personnel  in  the  Mess  Management  Specialist  School. 

The  two  CBT  groups  completed  the  course  in  significantly  less  time  than 
tin*  control  group,  and  the  No  .lob  Aid  CBT  group  performed  equally  as  well 
on  the  final  examination  as  the  control  group.  Although  the  Job  Aid  CBT 
subjects  performed  significantly  lower  on  the  examination  than  the  other 
CBT  and  control  groups — a finding  which  was  unexpected — they  still  did 
well,  having  a mean  test  score  of  87  percent.  This  suggests  that  low 
aptitude  students  can  profit  from  individualization,  which  greatly  reduces 
training  time,  and  still  maintain  a high  level  of  competency. 

At  this  point,  it  should  be  noted  that,  concurrently  with  our  initial 
survey  of  the  MS  school  recipe  conversion  area,  the  school  officials  them- 
selves were  becoming  d isenchanted  with  the  school's  curriculum.  Thus,  for 
a number  of  reasons,  our  interest  may  have  .acted  as  a stimulus  for  revision 
of  the  curriculum.  At  the  time  of  the  initial  survey,  trainees  in  the  math 
area  were  experiencing  a significant  failure  rate  (50%  scored  below  75  on 
the  area  final),  as  well  as  performing  poorly  in  the  lab  situation  (galley) 
which  is  the  school's  criterion  measure.  Accordingly,  the  two  CBT  courses 
were  designed  to  fill  the  need  for  remedial  math.  However,  by  the  time 
the  CBT  courses  had  been  written  and  students  were  ready  to  be  instructed, 
the  MS  school  had  revised  its  priorities  to  teach  criterion-referenced 
skills  of  recipe  conversion.  Because  of  this  change  in  emphasis  that 
occurred  in  the  interim  between  the  initial  survey  and  the  completion  of 
the  CBT  project,  it  was  necessary  to  compare  the  CBT  presentations  with  the 
revised  classroom  math  procedures.  As  a result,  performance  differences 
may  have  been  nullified  due  to  improved  classroom  revisions. 

Further,  this  study  did  not  consider  the  scheduling  problems  that  may 
be  involved  in  instituting  CBT  while  still  having  to  maintain  galley  train- 
ing, which  requires  40  students  simultaneously  in  order  to  fully  utilize 
the  galley  facilities. 


RECOMMENDATIONS 

The  following  recommendations  are  made: 

1.  The  use  of  the  job  aid  should  be  taught  separately  from  theory  on 
recipe  conversion.  If  the  student  is  not  forced  to  learn  two  new  things 

at  once,  such  training  may  result  in  both  time  savings  and  improved  scores. 

2.  The  CBT  lesson  that  uses  mathematical  solutions  should  be  used  as 

a dr  1 1 1-and-prac t ice  remedial  tool  by  the  MS  school,  given  the  availability 
of  1M.AT0  or  another  CBT  system. 
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